COUPLING AND TRACKING OF REGIME-SWITCHING MARTINGALES 



SAUL D. JACKA AND ALEKSANDAR MIJATOVIC 

Abstract. This paper describes two explicit couplings of standard Brownian motions B and V, which 
naturally extend the mirror coupling and the synchronous coupling and respectively maximise and min- 
imise (uniformly over all time horizons) the coupling time and the tracking error of two regime-switching 
martingales. The generalised mirror coupling minimizes the coupling time of the two martingales while 
£N| ■ simultaneously maximising the tracking error for all time horizons. The generalised synchronous cou- 

O | pling maximises the coupling time and minimises the tracking error over all co-adapted couplings. The 

1 proofs are based on the Bellman principle. We give counterexamples to the conjectured optimality of 

the two couplings amongst a wider classes of stochastic integrals. 

P? 

Let (fl, (^ r t)t>o ) P) be a filtered probability space that supports a standard (J r t)-Brownian motion 
B = (B t ) t >o and let 

V := {V = (Vt)t>o ■ V is an (J 7 t )-Brownian motion with Vq = 0} 

o 

be the set of all (J^-Brownian motions on this probability space. It is well-known that for any time 
horizon T > the Brownian motion in V which minimises the probability that the processes X = x+B 
Q\ " and Y(V) = y + V couple after time T (for any starting points x,y G R), i.e. the Brownian motion 

^ \ that solves the problem 



X 
S3 



1. Introduction 



minimise P [t (X - Y(V)) > T] over V eV, 

where tq(X — Y(V)) := inf{i > : Xt = Yt(V)}, is given by the mirror coupling V = —B. Furthermore 
it is easy to see that the Brownian motion which minimises the tracking error of Y(V) with respect 
to the target X at time T, i.e. solves 



minimise 



{X T - Y T (V)f 



over V £ V, 



is given by the synchronous coupling V = B. This paper investigates the following generalisations of 
these questions. 

1.1. Problems. Let Z = (Zt)t>o be an (J^-Feller process, i.e. a Feller process on our probability 
space, which is (J^)-Markov. Let the state space E of Z be a subset of a Euclidean space R rf for 
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some d G N. For real Borel measurable functions crj : E — >■ R, % = 1, 2, define the stochastic integrals 
X = (X t ) t > and y(V) = (Y t (V)) t > by 

(1.1) X t :=x+ I' a 1 (Z s )dB s and F f (V) := y + /* a 2 (Z s ) dV s , 

Jo Jo 

where i,i/£l and V£V. Throughout the paper we assume that for each starting point the process 

Z is a semimartingale (i.e. has cadlag paths, is non-explosive and without instantaneous states) and 



(1.2) E f af{Z s )ds <oo for all t > 0, i = 1, 2. 

Jo 



This implies that the processes X and Y(V) in (jl.ip are well-defined true martingales (e.g. see [91 
Cor IV. 1.25]). In the case the state space E of Z is embedded in a multidimensional space, a natural 
choice for the volatility functions a\ and 02 are the projections resulting in <J\{Z) and &2{Z) being 
coordinate processes of Z in M. d . Furthermore, to avoid degenerate situations, we assume throughout 
the paper that (|o"i| + |c"2 1 ) (^) > for all z G E. The class of stochastic integrals in (jl.ip . with the 
integrand Z typically a jump-diffusion (i.e. a Feller process), arises frequently and is of interest in the 
theory and practice of mathematical finance in the guise of stochastic volatility models (see e.g. [3]). 

We are interested in the "distance" between the two processes X and Y(V) for any V G V. In other 
words we seek to understand how large and small the following quantities can be 

(1.3) E [<P(Xt — Yt(V))] and P [tq(X — Y(V)) > T] , 

where T > is a fixed time horizon, <p G C 2 (]R) is a convex function bounded from below that satisfies 
the inequality |<^(x)| < a\x\ p for all x with large modulus for some constants a > 0, p G [2,oo), and 
tq(X — Y(V)) := inf{t > : X t = Y t (V)} is the coupling time of the processes X and Y(V). Since 
V is an arbitrary (J^)-Brownian motion, the law of the difference X — Y(V) is in general not easy to 
describe. Therefore we cannot expect to be able to identify the quantities in (jl.3p explicitly. Our goal 
is to establish sharp upper and lower bounds for the expectations in (|1.3p . which hold for any choice 
of Brownian motion V £ V and are based on a natural generalisationis of the mirror and synchronous 
couplings of Brownian motions described in Section 11.21 More precisely, we are looking for Brownian 
motions V M , V s £ V such that the following inequalities hold for all V G V: 

(T) E [<f>(X T - Y T (V S ))] < E [<P(Xt — Yt(V))] < e[<P(X t - Y T (V M )) 

(C) P[t (X -Y(V M )) >T] < P [t (X — Y(V)) > T] < P \t (X — Y(V S )) > T 

where the generalised mirror (resp. synchronous) coupling holds for B and V M (resp. V s ). 

In Problems (T) and (C), the goal is not merely to prove the existence in an abstract sense of 
the integrators V M ,V S G V, but primarily to understand for which classes of (J r t)-Feller processes Z 
are the generalised mirror and synchronous couplings of Brownian motions, described in Section [1.21 
extremal in the inequalities of Problems (T) and (C). In particular, for the volatility processes Z 
with the property that the generalised mirror and synchronous couplings satisfy the inequalities above 
for all Brownian motions V G V, the following holds: maximising the coupling time of the stochastic 
integrals minimises the "convex distance" of the two processes and vice versa uniformly over all time 
horizons T > 0. 
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1.2. Results. In the setting of processes (jl.ip . it is natural to define generalised synchronous and 
mirror couplings of Brownian motions in the following way. Let the functions c/ , c// : E — > R be given 
by the formulae 

c/0) := sgn(a 1 (z)a 2 (z)), &n(z) := - sga(a 1 (z)a 2 {z)) 
for any zGE, and define the Brownian motions V 1 = (V t r )t>o and V 11 = (V^ 7 ) t >o in V by 

(1.4) V/ := f ci(Z s )&B s and V t " := [ c n (Z s ) dB s . 

Jo Jo 

Note that cjj = —cj and hence V 11 = —V 1 . It is clear from f)l .4[) that B and generalise the 
synchronous coupling of Brownian motions, while the pair B and V n extends the notion of the mirror 
coupling. A natural conjecture, based on the case where X and Y(V) are Brownian motions, goes as 
follows. 

Conjecture. For any (J-i)-Feller process Z and V £ V, the inequalities in (T) and (C) are satisfied 
by V s = V 1 and V M = V 11 = -V 1 . 

1.2.1. The conjecture fails in the class of general (J 7 t)-Feller processes. Let the Feller process Z, with 
state space E := (0,oo), be defined as 

(1.5) Z t := z M t , where M t := exp(B t - t/2) and z > 0, 

and the volatility functions o~\,o~i : E — > R given by o~i(z) := —iz for any z G E and i = 1,2. 
The corresponding candidate extremal Brownian motions V 1 and V 11 , defined in (jl.4|) . are in this 
case given by the classical synchronous V 1 = B and mirror V 11 = — B couplings. The fact that 
Mt = 1 + Jq M s dB s yields Jq ai(Z s )dB s = —izo{Mt — 1), for i = 1,2, which in particular implies the 
following for all t > 0: 

(1.6) X t -Y t (V I )=x-y + z (M t -l) and X t — Y t (V H ) = x — y — 3z Q (M t — 1). 

Fix a time horizon T > and note that, since (jl.6p implies the supports of the random variables 
X T - YxiV 1 ) and X T - Y T (V n ) are given by 

supp (X T - YtCV 1 )) = {x - y - z Q ,co) and supp (X T - Y T (V n )) = (-oo, x - y + 3z ), 

any non- negative non-zero convex function (p : R — > R that satisfies the assumptions of Section 11.11 
with support (i.e. the closure of </> _1 (0, oo)) contained in the half-line {x — y + 3zo 5 oo)> clearly yields 

= E [0 (X T - Y T (V n ))) <E[4> (X t - Y T {V 1 ))] . 

Hence the tracking part of the conjecture fails for Z = zqM. 

Assume that the starting points in (|l.lj) satisfy x — y < — 3zo and define the stopping time 
t := inf{t > : B t — t/2 = log(l — (x — y)/zo)}. Note that the representations in (I1.6P imply 
P [t (X - Y(V H )) = oo] = 1 and P [t (X - YiV 1 )) > T] = P [r > T\ < 1 for any time horizon 
T > 0. Therefore the coupling part of the conjecture also fails: 

P [t (X - YiV 1 )) > T] < P [t (X - Y{V H )) > T] = 1. 
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1.2.2. The generalised mirror and synchronous couplings are optimal if Z is a continuous-time Markov 
chain. Throughout the rest of the paper let Z denote a continuous-time (J^-Markov chain that 
satisfies the assumptions in Section 11.11 (i.e. Z is an (J-i)-Markov semimartingale with a countable 
state space E in R d for some d G N). The following assumptions on the semigroup P of the volatility 
chain Z implies that the expectations in the inequalities of (T) are well-defined (see Section [3|): 

(1.7) VzGE: (P T (| £ Ji| p + |a 2 | p ))(z) <oo. 

Theorem 1.1. Let an (T^-Markov chain Z be as above. If Z also satisfies (\1.7\i , we have 

E [<P(X T - YriV 1 ))] < E[c/)(X T -Y T {V))} < E [<f>(X T - Y T {V H ))] foranyVeV. 

The integr ability condition in (|1.7p is not necessary for the solution of Problem (C). 

Theorem 1.2. Let an {J-t)-Markov chain Z satisfy the assumptions in Section \l.l\ Then 

P [t (X - Y(V H )) > T] < P [t (X - Y{V)) > T] < P [t (X - YiV 1 )) > T) for any V £ V. 

Remarks. (i) The function cj = —cjj, and hence the Brownian motions V 1 = —V 11 , that feature 
in the solution of Problems (T) and (C) depend neither on the maturity T nor on the precise 
form of the convex cost function 4>. Note also that essentially no restriction on the volatility 
functions o~\ and a 2 in the stochastic integrals in (jl.ip is necessary, for the two theorems to hold. 
Furthermore, the assumptions in Theorems 11.11 and 11.21 place no restrictions on the filtration 
(•^)t>o; in particular (J r t)t>o need not be generated by the processes B and Z. 

(ii) Brownian motion V 1 (resp. V 11 ) is chosen to minimise (resp. maximise) at each moment in time 
the Radon-Nikodym derivative of the quadratic variation of the process X — Y(V) over the set 
V. It is clear that V 1 and V can also be defined for much more general integrands than the 
ones considered in (jl.ip and that the generalisations will still be locally extremal. 

(iii) Section [3.21 shows that local maximisation/minimisation of the Radon-Nikodym derivative men- 
tioned in item (jn]) is also globally optimal in a non-Markovian setting in the special case of 
the quadratic tracking (i.e. where the cost function is (j)(x) = x 2 ). Section 14.31 establishes a 
coupling result, analogous to Theorem 11.21 in the case where the volatility processes are time- 
inhomogeneous but deterministic. However, Sections 11.2.11 and 15.31 show that the generalisations 
of Theorems 11.11 and 11.21 do not hold for general (J-^-Feller processes. 

(iv) The key fact, established in Lemma f2.31 that enables us to prove Theorems 1 1 . 1 1 and 1 1 . 2 1 is that the 
chain Z is in fact independent of the driving Brownian motion B (see Section I2.3H . It is therefore 
natural to ask whether the results in the theorems above hold for a general (J-t)-Feller process 
Z, which is independent of B. The example in Section 15.31 shows that Theorem 11.11 cannot be 
generalised even if such independence is assumed. 

(v) The volatility functions a\ and 02 are typically distinct, which makes the maximal coupling time 
tq(X — YlV 1 )) finite. Hence the upper bound in Theorem ll.2l is non-trivial (i.e. smaller than 1). 

(vi) Recall that sgn(x) is 1 if x > and — 1 if x < 0. In the setting of Theorems 1 1 . 1 1 and 1 1 . 2 1 the choice 
of sgn(0) in {1, —1} can be arbitrary, since by [SJ Prop IV. 1.13] it influences neither the laws of 
the processes </>(X -Y^ 1 )), <j>(X - Y(V U )) nor of the variables t (X -Y^ 1 ))^^ -Y{V h )). 
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(vii) In pQ the authors establish an inequality, analogous to the first inequality of Theorem 11.11 in 
the case X and Y(V) are solutions of driftless SDEs. A related inverse question to the tracking 
problem is studied in [7]. A general reference on the theory of coupling is given in [6]. 

(viii) The seminal paper [4] introduced regime-switching models to economics and finance. Since then, 
regime-switching models have found a plethora of applications in areas as diverse as macroeco- 
nomics, term-structure modelling and option pricing (see e.g. [5 J and the references therein). 

1.3. Structure of the paper. Sections 12.11 and 12.21 state two well-known lemmas that allow us to 
relate the coupling inequalities above to problems in stochastic control. Section 12.31 proves that the 
(J-t)-Markov chain Z and the Brownian motion B are independent. Sections [3] and 13.11 prove Theo- 
rem ll.ll Section [3721 discusses Problem (T) in a non-Markovian setting and establishes a generalisation 
of Theorem 11.11 in the case of a quadratic cost function. In Sections HJ 14.11 and 14.21 we establish The- 
orem 11.21 Section 14.31 proves an analogue of Theorem 11.21 in the case the volatility processes are 
time-inhomogeneous but deterministic. Section [5] discusses four counterexamples to the Conjecture 
above in the case where certain assumptions of Theorems 11.11 and 11.21 are violated. Appendix [A] 
contains the proofs of Lemmas 12.11 and 12.21 of Section [2j 

2. Preliminaries 

2.1. The set of Brownian motions on a probability space. Without loss of generality we may 
assume that the probability space (O, (J-t)t>o, J~, P), where the (J-^-Brownian motion B and the chain 
Z in (jl.ip are defined, supports a further (J^)-Brownian motion B 1 - G V, which is independent of B. 
If this were not the case, we could enlarge the probability space and note that this only increases the 
set V of all (J^-Brownian motions. Since the extremal Brownian motions V 1 , V n in Problems (T) 
and (C) are constructed from B and Z alone, they must also be extremal in the original problem. 
We shall henceforth assume that B 1 - G V exists. Any V G V and the process X — Y(V), which plays 
a key role in all that follows, therefore possess the following representation. 

Lemma 2.1. For any V G V there exist [Ft) -Brownian motion W G V and C = (Ct)t>o, such that 
W and B are independent, C is progressively measurable with — 1 < Ct < 1 for all t > P-a.s., and 
the following representations hold: 

(2.1) V t = f C s dB s + f\l - C 2 ,) 1 ' 2 dW s , 

JO Jo 

and X — Y(V) = R(V), where RiV) = {Rt{V))t>o is given by Rq(V) = r := x — y and 

(2.2) R t (V) := r + f {a x {Z s ) - C s a 2 (Z s )) dB s - I (1 - C 2 s ) 1 ' 2 a 2 {Z s ) dW s . 

Jo Jo 

Remarks. (i) The proof of l2.1l in Lemma [2. II is a well-known representation for a Brownian motion 
V G V in terms of B. For completeness and because of the importance of the representation 
in (|2,2p . which follows directly from (|2.ip . the proof of Lemma |2. II is given in the appendix (see 
Section IA.1|) ; it is this proof that requires the existence of B 1 - G V independent of B. 
(ii) Note that W and B in Lemma [2. II are independent, but the process C may depend on either (or 
both) Brownian motions B,W. 
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2.2. Q-matrices, related operators and martingales. Let Q denote the Q-matrix of the 
continuous-time (J-t)-Markov chain Z. We define the action of Q on the space of bounded functions 
on E in the standard way: for a bounded g : E — > R, let 

Qg : E — > R be given by the formula (Qg)(z) : = Q(z, z')g(z'), 

z'eE 

since the series converges absolutely for every z£E, 

Let the function H : E x R ->■ R satisfy the assumptions: H(-, z) G C 2 (R) and iT(r, •) : E ->• R is 
bounded for any r G R. Then, for any c £ [—1,1], we define C C H : E x R — >• R by the formula: 

(2.3) (£ c #)(r,z) := \{al~ c2a x a 2 + a\) (z)^(r,z) + {QH(r,-))(z). 

The operator £ c is closely related to a generator of the process (R(V), Z) and will play an important 
role in the solution of the stochastic control problems. 

The next lemma describes a class of martingales related to the chain Z. 

Lemma 2.2. Let F : R + xRxE — >• R be a bounded function, such that for any z£E the restriction to 
the first two coordinates F(-, -, z) : R + x R — > R is continuous. Assume that the generator Q satisfies 

(2.4) sup{-Q{z,z) : z G E} < oo. 

-Lei [/ = (Ut)t>o be any continuous semimartingale, adapted to the filtration (Ft)t>o- Then the process 
M u = (Mf ) t > , given by 

Mf := Yl [F(s,U 8 ,Z 8 )-F(s,U 8 ,Z 8 -)]- (QF(s,U s , -))(Z S .) ds, 

0<s<t J ° 

is a true (J~t, P z ) -martingale for any starting point z G E. 

Remarks. (i) The key point in Lemma 12.21 is that we do not assume that the process (U, Z) is 
Markov, since all that is required of U is that it has continuous paths and is adapted to the 
underlying filtration on the original probability space. This fact plays a crucial role in the solution 
of our optimisation problems, as it allows us to eliminate all the (suboptimal) non-Markovian 
couplings of the Brownian motions V and B, the laws of which are not tractable. 

(ii) Assumption (|2.4p on Q is equivalent to stipulating that Q is a bounded linear operator. This is 
clearly satisfied when the state space E is finite. 

(iii) The result in Lemma 12.21 is well-known. Because of its central role in the proofs of Theorems ll.il 
and ll.2[ proof of Lemma 12.21 is given in Appendix IA.2I 

2.3. (J^-Brownian motion and continuous-time (J^-Markov chain are independent. Intu- 
itively, the independence of the chain Z and a Brownian motion W G V follows from the fact that any 
(J-t)-martingale of the form (tp(Zt,t))t>o, where tp is a real function defined on the product E x R + , is 
equal to the sum of its jumps minus an absolutely continuous compensator and therefore has constant 
covariation with any continuous semimartingale adapted to (J~t)t>o- The key fact underpinning this 
argument is that Z is a Markov process on the filtration (Ft)t>o (see Section T5.2I for counterexamples 
to Theorems II. II and I1.2I when this assumption is relaxed) . 

Lemma 2.3. An (Ft)-Markov chain Z is independent of any (Ft) -Brownian motion W in V. 
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Proof. We first show that the random variables Wt and Zt are independent for any T > 0. Let 
the functions / : R — > R and g : E — > R be bounded and measurable with / suitably smooth. 
We need to establish the equality E[f (Wt)9{Zt)] = E[/(Wt)]E[p(Zt)]. Define the (J r t)-martingales 

M/ = (M/) t6[0)T] and A™ = (JVf ) t6[0jT] by 

M/ := E[f(W T )\T t ] and JV/ := E[g(Z T )| F t }. 

Note that it is sufficient to prove that the product M^N 9 = {M{ Nf ) te[0 . is a martingale since in 
that case we have 

(2.5) E[f(W T )]E[g(Z T )} = M/JV* = E[M^N T ] = E[f(W T )g(Z T )}. 

Now M/ = {P^_J)(W t ), where P w is the Brownian semigroup, and hence M* is a continuous 
martingale. Similarly we have Nf = (PT-tg)(Zf), where P denotes the semigroup for Z, and hence 
Ito's lemma for general semimartingales [8] Sec II. 7, Thm. 33] and the Kolmogorov backward equation 
imply dNf = (Pr-tg)(Zt) — (Pr-tg){Zt-) — (Q(PT-tg))(Zt)dt (Q denotes the generator matrix for Z). 
In particular, the quadratic variation of N 9 is equal to the sum of its jumps, i.e. the continuous part 
of the process [N 9 ,N 9 ] is almost surely zero. Hence the continuity of M$ and [3 Sec II.6, Thm. 28] 
imply that the covariation satisfies d[M^ , N 9 ]t = 0. Therefore, by the product rule, the infinitesimal 
increment of the process M^N 9 equals 

d(M t f N?) = N?_dM t f + M{_dNf + d[M f , N 9 ] t = N?dM t f + M t f dN? 

(the subscripts t— can be change to t since is continuous), making M* N 9 a martingale, since both 
M* and N 9 are bounded martingales, and equality (|2.5[) follows. By an approximation argument and 
the Dominated Convergence Theorem we conclude that f|2.5[) holds for arbitrary bounded measurable 
functions / and g and the independence of Wt and Zt follows. 

To prove independence of random vectors (W^, ■ ■ ■ , Wt n ) and (Z tl , . . . , Z tn ) for any n E N and a 
sequence of times = to < t% < ■ ■ ■ < t n , pick any bounded measurable functions / : R n — > R and 
g : E n — > R and define recursively the functions fk ■ R fcvl — > R and gk : E fcvl — > R for k = n, . . . , 0, 
which are again bounded and measurable, by /„ := f,g n '■= g and 

f k ^(W tl ,. . . , W tk _,) := E[f k (W tl ,. . . , WiJIJVJ, 9k-i(Z tl ,. . . , Z tk _ x ) := E[g k (Z tl ,. . . , Z tfc )|^ fc _J. 

Note that /o and 50 are constant functions. Equality (12.5f) applied to the bounded measurable func- 
tions x i y /(W*i, • • • > Wt„_i) x) and z i-> g(Z tl , . . . , Z tn _ 1 , z) shows that the following conditional 
expectation factorises: 

E[f(W tl ,. . .,W tn )g(Z tl ,. . . ,Z tn )|7i B _ l ] = / n -i(W tl , . . . , W^)^!^, . . 

Therefore, by iteration and the tower property, we see that the following holds 

E[/(W tl , . . . , W tn )g(Z tl ,...,Z tn )} = f ogo = E[f(W h ,. . . , W tn )]E[g(Z tl ,...,Z tn )}, 

Since / and g were arbitrary, the processes W and Z are independent. □ 

It follows from Lemma 12.31 that an (J-t)-adapted volatility process, given by a strong solution of an 
SDE, cannot be approximated pathwise by a continuous-time (J r t)-Markov chain. 
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Corollary 2.4. Let Z' be an (J-t)- adapted Feller semimartingale, which solves a scalar SDE with 
Lipschitz drift and diffusion coefficients f/,,o~ such that a > c > 0. Then there exists no sequence of 
continuous-time (J-t) -Markov chains that converges to Z' almost surely on compacts. 

Proof. The process W = (W t )t>o, where W t := p a (dZ' s -ii(Z' s )dt) / a(Z' s ), is an {J-t )-adapted continuous 
local martingale with [W, W]t = t. W is therefore an (J-i)-Brownian motion (by Levy's characterisation 
theorem) and Z' is a strong solution of the SDE dZ' t = ^(Z' t )dt + o~(Z' t )dWp By Lemma 12.31 an Y 
sequence of continuous-time (J-t)-Markov chains is independent of W and therefore also independent 
of Z'. Therefore, since Z' is non-deterministic, the sequence cannot converge to Z' almost surely on 
compacts. □ 



3. Tracking 

In this section we consider the problem of tracking X by the process Y(V), defined in (jl.ip . where 
the control is being exercised solely by choosing the driving Brownian motion V. Recall that the 
tracking criterion is given by a convex <f> £ C 2 (R) bounded from below, which satisfies |<^(x)| < a\x\ p 
for large x and some a > 0, p £ [2, oo). More precisely, for T > 0, we consider the following problems: 

minimise E [<P(Xt — Yr(V))] over V £ V, 
maximise E [(/)(Xt — Yt(V))] over V £ V. 

Theorem 3.1. Let the Brownian motions V 1 and V 11 be as in (|1.4p and assume that Z satisfies (|1 .7[) . 
Then for any positive T we have 

(3.1) inf E [0(X T - Y T (V))} = E [0(X T - K r (V J ))] , 

vev 

(3.2) supE[0(X T -y T (y))] = E U>(Xt — YriV 11 ))] . 

v&v 

In this section we prove Theorem 13.11 which clearly implies Theorem ll.lt an d hence solves Prob- 
lem (T). The proof of Theorem l3.1l is based on Bellman's principle, a martingale verification argument 
and an approximation scheme. The first stage consists of "approximating" Problems (|3. 1 1> - f|3. 2 [) . More 
precisely, we proceed in two steps: we first introduce a stopped chain Z n and, in the second step, the 
stopped process R K,n (V). 

To this end let U n C n £ N, be a family of compact subsets such that U n ^U n = M. d and 
U n C U° +1 , for all n £ N, where U° +1 denotes the interior of U n +i in M. d . For each n £ N, define a 
stopping time r n and the stopped (J 7 t)-Markov chain Z n by 

(3.3) Z r t l := Z tATn , where r n := inf{t > : Z t £ E \ U n } (inf = oo). 
Hence, Z n is an (J-i)-Markov chain with the state space E and a Q-matrix Q n given by 

(3.4) Q n (z,z r ) = I Un (z)Q(z,z'), z,z'£E, 

where Ii.x denotes the indicator function. In particular, since U n is compact and hence U n n E must 
be finite, Q n satisfies assumption (|2.4p in Lemma 12.21 Since the chain Z has no instantaneous states 
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by assumption, the sequence of positive random variables (r„) ng N is non-decreasing and the following 
holds 

:= lim T n = oo P 2 -a.s. for any z G E. 



Hence, we can extend the definition in (J3J3J) in a natural way to the case n = oo by Z°° := Z. 
Fix a large K > and define, for any V G V, the stopping time 



t k (V) :=inf{i>0: 1^(^)1 > K} (inf0 = oo), 

where R(V) is given in (|2.2p . The stopped process of interest R K > n (V) = (R?' n (V)) t > Q can now be 
defined by 



(3.5) 



R?> n (V) :=R tATnArK{v) (V). 



For given <f) and T > as in Section [T] and any K G (0, oo) and n £ NU {oo}, consider the problems 



(3.6) 
(3.7) 



minimise E 
maximise E 



<t>(R%> n (V)) 
4>(R^' n (V)) 



over V £ V, 
over V £ V. 



By Lemma 12 . 3|. the processes {R{V l ),Z) and (R{V n ),Z) are Markov. Therefore we can define the 
candidate value functions VKn^^Kn : ^ x ^ x [0>^1 ~~ * 



(3.8) ^ n (r,z,t):=E r , z ^(R^V 1 )) 



and 



for Problems (|3.6p and (|3.7p by 



respectively. Note that by definition we have n( r > z ' ^) = ^if n( r > z ' ^) = < M r ) if r G R \ ( — K, K) or 
z£t \ U n . 



Lemma 3.2. For any K £ (0, oo) and n £ NU {oo}, i/ie functions and n> defined in (13. 8|) . 

/iaue i/ie following properties. 

(i) For all r G R, z£E and t G [0, T], there exists a constant £ £ R, suc/i that 

t < ^] n (r, z,t),Tp^(r, z,t) < max{</>(max{if,r}),</>(min{-if,r})}. 

(ii) For each z £ E we have ^] n {-,z,-),^l(-,z,-) £ C 2,1 (R x (0,T]). 

(in) For any r £ R, z £ E and t £ (0,T] 7 t/ie derivatives satisfy the following inequalities: 



(3.9) 



(3.10) 



(r,3,(), 



9r 



(r,z,t) < max{(j)' (max{K,r}), —(j)\mm{—K,r})}, 



dr 2 



Proof. Part (jl]) follows from ()3.8[) and the properties of <fi. To prove that ip^ n is differentiable in r, 
define S := i?f '"(V 7 ) — i? ' n (y 7 ) and note that its distribution does not depend on the starting point 
of R K ' n (V I ). Since G C 2 (R), La grange's mean value theorem implies that, for any small h > 0, 
there exists a random variable such that 



(3.11) 



{r + h + S)-^{r + S) = H'(r + t s , h ) and G (5, /i + 5). 



10 



SAUL D. JACKA AND ALEKSANDAR MIJATOVIC 



Since \S\ < K almost surely and r is fixed, the continuity of 4>' yields that the random variable 
\4>'(r + £s,h)\ is bounded above by a constant. Equation (|3.1ip . almost sure convergence of £s,h to 5, 
as h — > 0, and the Dominated Convergence Theorem imply that ip^P n (-, z, i) is differentiable in r and 



(3.12) 



Furthermore, the convexity of <f> and f)3. 12|) yield the first inequality in (|3.9p . An identical argument 
applied to the function ip^ 1 ^-, z,t) implies its differentiability in r and yields f)3. 9j) . 

Since <fi" is continuous by assumption, we can apply an analogous argument to the one above, now 
using formula (|3.12p instead of (|3.8p . to conclude that the functions i/j^ n (-, z,t) and V^n('> are 
in C 2 (R) with 



K,n 



dr 2 



(r,z,t) 



<mf' n (^)) 



-^(r,z,t) = E r , z [4>"(R?> n (V n )) 



The convexity of <f> now implies part (|m|) of the lemma. Differentiability of ip^p n (r, z, ■) in t follows 
from the smoothness of (f> and the standard properties of Ito integrals. □ 

Pick a function F:txEx[0,T)-fl such that F(-, z, •) G C 2 ' X (R x [0, T)) for each z G E, and for 
each r G R, i G [0, T) the restriction to the second coordinate F(r, -,t) : E — > R is bounded. Then for 
any constant c G [—1, 1] we define the function tC c F : R x E x [0,T) — > R by the formula: 

dF 



(C c F(;;t))(r,z)+—(r,z,t), 



()C c F)(r,z,t) = 
where the operator C c is as defined in ()2.3p . 

Lemma 3.3 (HJB equation). Let n G N one? i^T G (0,oo). T/ien i/ie functions 



F^\r,z,t):=^ n (r,z,T-t) and F^(r, z,t) := ^(r, z,T - t), 

(see (|3,8p for the definition of ij$ n and ip^l) sa ^ s /?/ the HJB equations: 
for any triplet (r,z,t) G (—K,K) x (E n U n ) x [0, T) fsee (|3.3p /or £/te roZe o/ the set U n ) we have 



(3.13) 
(3.14) 



inf (]CF^) (r,z,t) = 0, 
ce[-i,l] V / 



sup (/C c F( /7 n (r,z,t) 

c€[-l,l] V 7 



0. 



Furthermore, if at least one of the conditions \r\ > K or z G E \ U n or t = T is satisfied, it holds 
(3.15) F {I \r,z,t) = F {II) {r,z,t) = <j>(r). 

Remark. Unlike Lemma 13. 2\ the proof of Lemma 13.31 depends on Lemma 12.21 and so requires the 
assumption n < oo. 



Proof. Note first that the definitions in (|3.8|) imply the boundary behaviour stated in (|3.15p . 

We now focus on the proof of f)3. 13j) . Recall that for any starting point z G E and t G [0, T), on the 
event {r n > t} we have Zf = Z t . The Markov property of the process (i?(F 7 ), Z) and the equality 
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in ()3.15p now imply 



<Ki#' n (y J ))|Ji 



= E ^</>(R%' n (V I ))I{ Tn<t } | Jij + E [</>(R?' n (V J ))I {Tn > t} \? t 
= 0«'"(^))/ {Tri<i} + ^ n (i?f "(W), Z?, T - i)I {r „> t} 

The following observations are key: 

• the quadratic covariation [R ''"'(V 1 ), Z n ' l ]t vanishes for all t > and i = l,...,d, where Z r ' 
is the i-th component of Z n (recall that we are assuming E C 



the chain Z n satisfies the assumptions of Lemma [2. 2 1 and hence the process M u = {MY)t&^,T\ , 
given by 

M t ■■= E [i>%{Rf' n {V%Z-,T - s)-^{Rf' n {V%Z^T - s) 

0<s<t 



- fiQn^niR^iV 1 ), ;T - s))(Z^_) ds, 

Jo 



where Q n is the generator of the chain Z n given in (|3.4p . is a true (J-f, P 2 )-martingale for any 
starting point z£E. 

(I) 



By Lemma |3.2| the function ip Kn possesses the necessary smoothness so that Ito's lemma for general 
semimartingales [8j Sec II. 7, Thm. 33] can be applied to the process (^nl^f '"!^^' %t ■> ^ ~ ^))te\o,T] j 
which is itself a bounded martingale. Since Q n (z,z') = Q(z,z') for any zGEfl U n , z'eE and on the 
event {t < r n } we have Z± = Z™ G U n , the pathwise representation of this bounded martingale implies 
that the following process N = (A r i ) tG [ ,T]; 



N t 



tAr n Ar K (V) 



-(|ai| - \a 2 \f(Z s 



dr 2 



-(R^iV^Z^T-s) 



+ m { K] n {R^ n {V I )^T-s)){Z s 



dt 



(R^iV^Z^T-s) 



ds, 



is a continuous martingale. The quadratic variation of N is clearly equal to zero and hence Nt = for 
all t G [0, T] and starting points (r, z). Since the process (R K,n (V I ), Z) visits a neighbourhood of any 
point in the product (-K, K) x (Efl U n ) on the stochastic interval [0, T A r n A t k (V)] with positive 
probability, for all (r, G (-K,K) x (En C/ n ) x [0,T) we must have: 



1 



(3.16) j(|oi| - M) 2 (*, , y/ 2 



'^(r,z,T-t) + (<W-g? ( r> -,r-t))(z) 



0* 



r,z,T - t) 



0. 



To prove (|3.13p . observe that (|<7i| — | ct2 | ) 2 = inf ce r_i i](af — 2ccriO"2+o"2)- Then (|3.10j) of Lemma l3.2l 
implies that 



(a 1 2 -2c ( T 1 a 2 + C T 2 2 )(z)^4^(r,z,r-t) > (M - |a 2 |) 2 (z) 



r,z,T - t) 



„ n V' > "1 - 1 "V — \ -I- \"/ o o 

for any c G [-1, 1] and each (r, z, i) G (-if, if) x (E D £/ n ) x [0, T). This inequality and identity t|3.16j) 
imply (|3.13|) . The proof of (|3.14|) is analogous and therefore left to the reader. □ 
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3.1. Proof of Theorem 13.11 Pick V G V and, for any t G [0,T], define Brownian motions V 



it 



)s>0 



G V and V Ht = (y 3 m ) a >o G V by 



(3.17) V s 



it 



V s 

v t + vj 



if s < t, 
if s > t, 



and V, 



J7t 




if s < t, 
if s > t, 



where V , V are given in (jl.4p . In other words, for each t > 0, the Brownian motions V 11 and 
V nt are arbitrary (but fixed) up to time t and have increments equal to those of the candidate 
optimal Brownian motions after this time. We now consider two Bellman processes (B( (^))te[o,T] an d 
{Bl 1 '(V))te[o,T]> associated to Problems (I3.6p - (I3.7|) . given by 



(3.18) 



Bl(V) := E \<KR%' n (V H ))\F t ] and B» (V) := E \ct>(R^ n (V m ))\T t 



The definitions in f j 1 . 4 [) of V , V 11 , together with Lemma 12.31 imply that the processes (RiV 1 ), Z) 
and (R(V ), Z) are Markov. The definition of the Brownian motion V 11 in f|3. 1T|) and the properties 
of the function ipjp„ therefore imply 



<P{R^ n {V H ))\F t 



= <KR% n {V))I {rn<t} + ^ n (K' n (v), Z?,T - t)I {Tn > t} 

This equality, together with a similar argument based on the definitions of V nt and ip^n' Y^ds the 
following representations for the Bellman processes 



K,n, 



B((V) = ^ n (R?' n (V),Z?,T-t) and B? (V) = ^HR?> n {V), Z?, T - t). 



(II) m K,n, 



By Lemma 13.21 we can apply Ito's formula for general semimartingales (see [3 Sec II. 7, Thm. 33]) 
to B J (y) and B n (V). Lemma 12.21 and inequalities (|3.9p imply that the local martingale parts of 
these path decompositions of processes B 1 (V) and B n (V) are true martingales. Therefore, the fact 
that the quadratic covariation [R K,n (V It ), Z n,l ] t vanishes for all £ > for each component Z n,t of 
Z n , together with Lemma |3.3[ implies that, for any V G V, B 1 (V) is a submartingale and B n (V) 
a supermartingale. Furthermore it follows from the discussion above and Lemma 13.31 that B^V 1 ) 
and B n (V n ) are martingales. This establishes the Bellman principle and solves the optimisation 
problems in (|3.6p and (|3.7p . Put differently, we have established the following inequalities for any 
starting points r G R, z G E, any K G (0, oo), n G N and all Brownian motions V G V: 



(3.19) 



HR^iV 1 )) 



< 



4>{R^ n {V)) 



< 



0(it£- n (O) 



II\ 



The final step in the proof of Theorem 13. II requires two limiting arguments. First, note that for any 



Brownian motion V G V the definition of the process Rp n (V) in (|33|) implies 

P, 



R%'°°(y) 



lim R^ n {V) 

ntoo 



z -a.s. 



for any starting points r G R and z G E. Furthermore, by Lemma 13.21 the random variables 



4>{R^ n (V)) are bounded in modulus by a constant uniformly in n G N. Therefore, the Dominated 
Convergence Theorem implies that the inequalities in ()3.19|) hold for n = oo. 
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For the second limiting argument, note that assumption (jl.7p and the semigroup property imply 
P t f(z) < oo for all t G [0,T], z G E, where / := |<ri|* + \a 2 \ p : E -> [0,oo) and p G [2,oo). The 
Kolmogorov equation Pt/ = Q J -ft/di + / in integral form now yields 

(3.20) E z f (M p + \a 2 \ p ) (Z t )dt < oo for z G E. 

JO 

Furthermore, it is clear from the definition of R '°°(V), for any V G V, that 

lim <j}(B^'°°(V)) = <f>(R T (V)) P rz -a.s. 

K— s-oo 

The following almost sure inequality is a direct consequence of the definition in (|3.5p 

i2*'°°(V0 < 5 := sup R t (V) for all if > 0. 

te[o,T] 

Since, by assumption, there exists a > such that \4>(r)\ < a\r\ p for all r G K, the following inequalities 
hold 

mT ,CO (V))\ < \<KS)\ < a\S\ P - 

The Burkholder-Davis-Gundy inequality [9, Thm IV. 4.1] applied to the martingale R(V) at time T, 
together with inequality ()3.20p . implies that \S\ P is an integrable random variable. The Dominated 
Convergence Theorem therefore yields the L 1 -convergence for <f>(R^ '°°(V)) — > (/)(Rt(V)) as K — > oo. 
By (|3.19p for n = oo, we obtain the following inequalities for any V G V: 

Er.MRHV 11 ))] = lim E r ^(4'°°(^))] > lim E^R^' 00 (V))} = E r>z [cj>(R T (V))] 

> lim E r> M^'°°(V I ))} = E r ,MRT(V 1 ))], 

K^oo 

concluding the proof of Theorem 13.11 □ 

3.2. Non-Markovian Tracking. The Markovian structure does not feature explicitly in the conclu- 
sion of Theorem 13. 1 1. but only in its assumptions. It is therefore natural to ask whether, under some 
additional hypothesis, Theorem 13 . 1 1 can be generalised to a non-Markov volatility process Z. In this 
section we argue that, for such a generalisation to hold for a large class of convex cost functions 4>, an 
underlying Markovian structure is in fact necessary but show that it is possible in the case <f>(x) = x 2 . 
Assume (in this section only) that the stochastic integrals X and Y(V) are given by 

(3.21) X t = x + f H s dB s and Y t {V) = y + /* J s dV s , 

Jo Jo 

for some progressively measurable integrands H = (H t )t>o and J = (Jt)t>o on (Q, (J r t)t>Q, P) 
and any V G V. As usual, we denote the difference of X and Y(V) by R(V) = X — Y(V). The 
extremal Brownian motions V 1 and V , defined in (jl.4p . can be generalised naturally by V t = 
J*q sgn(H s J s ) dB s and V t n = —V/. Hence, for any fixed V G V, we can define the Brownian motions 
V n and V nt as in (|3.17p . If the generalisation of Theorem 13 . 1 1 were to hold in this setting, the Bellman 
processes B 1 (V) and B n (V), defined in (|3.18p . would be a submartingale and a supermartingale, 
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respectively, for any V G V. We will focus on B^V), as the issues with B n (V) are completely 
analogous. Representation (|2.ip of V in Lemma 12. II and ito's formula yield 

(R T (V Jt )) = <j> {RoiV 11 )) + Mj> + 7- / 0" (i? s (F)) (tf 2 - 2C S F S J s + J 2 ) ds 

2 Jo 

+ \[ <f>" ( R s( yI ) ~ Mv 1 ) + RtOO) (\h s \ - | j s |) 2 ds, 

where M is a local martingale, which we assume to be a true martingale. The process B{ (V) = 
E \ [ cf){RT{V It ))\ Ft\ is a submartingale if and only if the conditional expectation E[B^(V) — Bl(V)\Ft], 
proportional to 

E [ [</>» (R^V 1 ) - RAV 1 ) + Rf(V)) - 4>" [R s {V l ) - R t {V l ) + Rt(V))] (\H S \ - \J S \) 2 ds 
+ // <P" (R S (V)) (H 2 S - 2C S H S J S + Jf) - <\>" {RsiV 1 ) - RtiV 1 ) + R t (V)) {\H S \ - \ J s \) 2 ds\ T t 

by the formula above, is non-neg ative for all < t < t' < T. Hence B^V) is a submartingale for 
general integrands J and H if (j)" does not depend on the state, i.e. when the cost criterion <j) is 
quadratic, and we obtain: 

Proposition 3.4. Let R{V) = X - Y(V), where X, Y(V) are as in (f3T2T|) , and T > 0. Then we have 
E [(X T - YriV 1 )) 2 } < E[(X T -Y T (V)) 2 } < E[(X T - Y T (V n )) 2 } foranyVeV. 

This proposition is consistent with an argument based on Ito's isometry: the variance of a stochastic 
integral is equal to the expectation of its quadratic variation and hence minimising/maximising its 
variance is equivalent to locally minimising/maximising the Radon-Nikodym derivative of its quadratic 
variation. Furthermore, it is also clear from the representation above that in the absence of an under- 
lying Markovian structure, for a general convex (f>, the process B (V) will not be a submartingale and 
hence, we believe, the strategy in Theorem 13.11 is not optimal for general non-Markovian integrands. 

4. Coupling 

In this section we consider the problems of minimising and maximising the coupling time of the 
processes X and Y(V) defined in (jl.ip . where the controller is free to choose the driving Brownian 
motion V in the integral Y(V) and the volatility is driven by a continuous-time (J^-Markov chain 
Z. Put differently, we seek sharp upper and lower bounds for the probability of the event that the 
coupling of X and Y(V) occurs after a fixed time T. The couplings are characterised by the stochastic 
extrema of the stopping time tq(X — Y(V)) := inf{£ > : X t = Y t (V)} (with convention inf = oo). 
More precisely, for any fixed T > 0, we consider the following problems: 

minimise P [t (X - Y(V)) > T] over V G V, 
maximise P [tq(X — Y(V)) > T) over V € V. 

Theorem 4.1. Let V 1 and V be as given by (|1.4p . Then for any T > we have 

(4.1) M v P[t (X-Y(V))>T] = P[t (X-Y(V h ))>T), 

(4.2) snpP[T (X -Y(V))>T] = P [r (X -Y(V 1 )) > T] . 
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In this section we prove Theorem 14, 1\ which clearly implies Theorem 11.21 an d hence solves Prob- 
lem (C) for a continuous-time (J-t)-Markov chain Z. The aim is to minimise and maximise the 
coupling time of the martingales X and Y(V) given in (jl.ip . Due to the symmetry in Problem (C), 
we may therefore assume without loss of generality that the starting points of the processes Xq = x 
and Yq(V) = y satisfy the inequality 

(4.3) x < y. 

The candidate value functions in Problems (I4.1h and (14.2j) will be functionals of the law of the 
Markov processes (R{V n ),Z) and (i?(U 7 ),Z), respectively, where R(V) is given in (12. 2D and the 
Brownian motions V 11 and V 1 are defined in (|1.4p . The first step in the proof of Theorem 14.11 is 
to localise Problems (|4.ip and ()4.2p . With this in mind, for any n G N recall definition (j3.3|) of the 
stopping time r n and the stopped chain Z n . Unlike in Section [31 in the case of coupling it is important 
to localise the process R(V) by stopping only the integrand. The process R n (V) := (R r t l (V))t>o is 
therefore given by 

(4.4) R?(V):=r+ F a^Z?) dB s - f o 2 (Z n s ) dV s , r < 0, 

Jo Jo 

where B is the fixed Brownian motion and V G V any Brownian motion on our probability space. As 
in the previous section, in this circumstance it is also natural to identify the limit (R°°(V), Z°°) with 
the process (R(V), Z). For n G NU {oo}, we define the first entry time of the process R n (V) into the 
positive half-line by 

(4.5) T + {R n {V)) := inf{i > : R?(V) > 0} (with inf = oo). 

The localisation procedure will allow us to reduce the problem to the case where the generator of 
the volatility chain Z is bounded, which will in turn make it possible to establish sufficient regularity 
of the candidate value functions and conclude that certain processes are true martingales (see Sec- 
tion [5TT]). The two Markov processes (R IIn , Z n ) and (R In , Z n ), which play a key role in the solution 
of Problems (|4.ip and (|4.2p , are defined by 

(4.6) R\ In := r + f Z H (Z r :) dB s and R\ n := r + f £/(2?) dB s , 

Jo Jo 

for any r < 0, where B and Z n are as above and the functions S//, : E — > K are given by 

(4.7) E n (z) := a^z) + S gn{a 1 {z)a 2 {z))a 2 {z) VzGE, 

(4.8) Ej(z) := a^z) - sgn(a 1 (z)a2(z))a 2 (z) Vz G E. 

Note that, according to our definitions, we have R n (V n ) ^ R IIn and R^iV 1 ) ^ R In for any n G N, 
since the Brownian motions V 1 and V n , defined in (jl.4|) . are given in terms of Z and not Z n . However, 
if we define the Brownian motions V In and V IIn by ()1.4j) with Z replaced by Z™, then the equalities 
= R Un and ii"(y 7n ) = R In hold. 
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The proof of Theorem 14.11 can now be carried out in three steps. First, we formulate a pair of 
"approximate" coupling problems (for each n £ N): 

(4.9) minimise P T)Z [r£(R n (V)) > T] over V £ V, 

(4.10) maximise P T)Z [r+{R n (V)) > T] over V £ V, 

for a fixed T > and any starting points r < 0, z 6 E. The following probabilistic representations for 
the candidate value functions of Problems (14. 9p and (14. lOf) play an important role in their solutions: 

(4.11) C { n n) (r,z,t) := P r , z [r+(R IIn )>t], 

(4.12) d I] (r,z,t) := P r , z [r+{R In ) > t] . 

The second step, described in Section 14.11 solves Problems (|4.9h and (j4.10j) Lemmas 14.21 and 14.31 
establish the necessary analytical properties of the candidate value functions Cri an d Cn \ which 
enable us to prove (see Lemma 14. 4p the optimality of the Brownian motions V IIn and V In . More 
precisely, the representations in (|4.1ip - (l4.12p are used to establish the required differentiability of 
the functions Q. and Cn\ which allows us to study the pathwise evolution of the corresponding 
Bellman processes. The optimality of V IIn and V In , established in Lemma 14.41 is a consequence of 
the non-positivity of the second derivatives and Jffi proved in Lemma 14.31 

The third step in the proof of Theorem 14.11 given in Section [4.21 applies approximation arguments, 
which establish the Brownian motions V n and V 1 as the solutions of Problems (|4,ip and (|4.2p . 

Finally, Section 14.31 discusses the issues that arise with a direct approach, based on the Dambis, 
Dubins-Schwarz theorem (see e.g. [SJ Thm V.1.6]), to the coupling problems in (|4.ip and (|4.2p . 

4.1. The stochastic time-change. Throughout this section we fix n £ N. Let £// : E — > R be as 
in (|4.7|) and note that our standing assumption (|o"i| + lo^lX 2 ) > implies S|j(z) > for all z £ E. 
Therefore, the stochastic time-change A 11 = (Aj. 1 )^, given by 

(4.13) 4':= f^hiZ^ds, 

Jo 

is a differentiable, strictly increasing process. Furthermore, the definition of Z n and (|4.13p imply that 
the almost sure limit lim^oo A\ = oo holds. Hence, the inverse E n = (£f 7 ) s > , defined as the unique 
solution of 

A^n = s, s > 0, also satisfies ^A^ 1 = ^ ^ or an * — ^' 

and is a strictly increasing process with differentiable trajectories. Since Z n is an (^)-Markov chain, 
it is by Lemma 12.31 independent of the (J^)-Brownian motion B in (|4.6p . Therefore the laws of 
the processes (R IIn , Z n ) and (r + B A n,Z n ) coincide, where B A u denotes the Brownian motion B 
time-changed by the increasing process A . 

Let T,j : E — > M be as in (|4.8p and assume further that |<7i|(z) ^ \a2\(z) for all z £~E. This implies 
the inequality ^(z) > for all z £ E. Define, in an analogous way to (|4.13p . the strictly increasing 
continuous time-change A 1 = (A() t >o and its inverse E 1 = (Eg) s >o, and note that the processes 
(R In , Z n ) and (r + B A i, Z n ) have the same law. We can now state and prove Lemma 14.21 
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Lemma 4.2. Fix n G N and pick any r < 0. Define the stopping time := inf{s : B s = — r} (with 
inf = ooj and recall that the function G(r, t) := P [r,? > i] , /or any t > 0, takes the form 

G(r,t) = 2N(~)-l, 



where N(-) denotes the standard normal cdf. 

(a) For any z G E the following representation holds: 

(i n \r,z,t) = E z [G^Al 1 )}. 

qA 11 ) d 2 C (II) (9<r (//> 

Hence the partial derivatives g r (r, z, t), J 1 "^ (r, z, t), Q t (r, t) exisi for r < 0, i > 0. 
(^oj Assume further that \ax\(z') ^ |cr 2 1 for all z' G E. TTien /or any zGEk have 

(P(r,z,t) = E z [G(r,4)] 
and i/ie partial derivatives 2, i), (r, z, t), -4g— (r, 2, t) exisi /or any r < 0, i > 0. 



Proof. We first establish (jaj). Recall the definition of the time-change process ^4 77 and its inverse 
introduced above and note that the following equalities hold almost surely by the definition of the 
stopping time t^: 

E^b = mf{£f : B s = -r} = M{t : B A u = -r} (with inf = oo). 

Therefore, since the processes (R IIn , Z n ) and (r + B A n,Z n ) are equal in law, so are the random 
variables T + (R IIn ) and E H B . since E n is a strictly increasing continuous inverse of A 11 , we have 

P r , z [t<4(R IIn )] = P^A^Kr?] 
(4.14) = E z [G(r, Al 1 )] . 

This, together with definition (|4.11j) . implies the representation of £n in part (jaj) of the lemma. 

The required differentiability of Q. in r follows from (|4.14p . along the same lines as in the proof 
of Lemma 13.21 An application of the Dominated Convergence Theorem, the mean value theorem and 

the smoothness and boundedness of the functions ^ and on a rectangle (r — e, r + e) x (0, oo) 

nA 11 ) 

for any fixed r < and small e > 0, such that e + r < 0, together imply the existence of q t (r, z, i) 

) 

c*r 2 



and —$;-(r,z,t 



The differentiability of Cn i n t is more delicate as it is intimately related to the integrability of 
the chain Z n and the unboundedness of the function We start with the following observation. 
Claim. The stopping time r n , defined in (j3.3[) . is a continuous random variable and 

(4.15) E z [/ {Tn < s} Sf 7 (Z Tn )] < oo for any z G E and s > 0. 

Since P z [r n > t] = P z [Z™ G U n fl E] , the continuity of r n follows (the definition of the sets U n is given 
above equation (|3,3p ). To prove (|4.15p . note first that the assumption in (|l.'2p and definition (|4.7|) 
imply E z T,'j I (Z s )ds < oo for all z G E and t > 0. If E z T,'j I (Zt ) = oo for some to > and z G E, then 
the Markov property, the irreducibility of the chain Z and the positivity of Y> 2 n > together imply the 
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following equalities for any u > 0: E z Z to+u = E z [E[E J7 (Z to )| T u ]] = E z [E[S^(Z t[) )|Z n ]] = oo. Hence 
E z J T,'j I (Z s )ds = oo for any t > to, which contradicts assumption (jl.2p . This contradiction implies 

(4.16) E z [E 7/ (Z t )] < oo for any z G E and t > 0. 

Since Ej 7 > 0, by definition we must have {QTr II ){z) G (—00,00] for all z G E. The backward 
Kolmogorov equation implies the equality 

/ E z [(QE/ / )(Z rt )] du = E z [E? 7 (^)] - E?,(z), for all z G E and t > 0, 

which, together with (|4.16p and the irreducibility of the chain Z, yields |(QE| 7 )(^)| < 00 for all z G E. 
We therefore get 

(4.17) IIQnS//||oo :=sup|(Q n S /7 )(z)| < 00, 

zGE 

since, by definition (]3.4p . there are only finitely many states zGE, such that {Q n Y? n )(z) ^ 0, and for 
each of those states we have {Q n Y> 2 n )(z) = (QE 77 )(^). 

Let E 77 : E — > R + be defined by E 77 (z) := I{zetu n r<E}'^ii( z ) f° r an Y -2 G E and observe the equalities: 

hrn^sj^n ( z r n ) = E 77 (Z sATn ) = E 77 (Z™) for any s > 0. 

2 

Hence, to prove (|4.15p . we need to show E Z E 77 (Z™) < 00 for all states z G E and times s > 0. Recall, 
from the definition of Q n in (|3.4|) , that Q n is a bounded operator on the Banach space £oo (E) of bounded 
real functions mapping E into R. Let ||Q n ||oo < 00 denote its norm and recall that the norm satisfies 
1 1 Qn 1 1 00 < || Qn || So f° r an A: G N. We can therefore use the exponential series to define a bounded 
operator exp(sQ n ) and express the semigroup of Z n as follows: E 2 E 77 (Z™) = ^exp (sQ n ) E 77 ^j (z). 
Hence, by (|4.17p . we find 

^ v^ 2 /- \ 1 (s||Qn||oo)* ^2 11 

< E 77 (z) + s2__^ + WQn^nWoc <oo, 

for all z G E and s > 0. This implies ()4.15j) and proves the claim. 

In order to prove that Cn is differentiable in time, fix t > 0, r < 0, z G E and, for any At > 0, 
define the random variable 

D At (r, z, t) := [G(r, Ag M ) - G(r, A* 1 )] / (>4& At - Af ) . 

Since t > (resp. At > 0), we have A 1 / > (resp. (A^ At - A 1 ^) > 0) P 2 -a.s. Note also that the 
random variable |.D A t(r, z,t)\ is bounded by a constant uniformly in At > 0. This follows from the 
existence of a uniform bound on ^§-(r, •) in the second variable for any fixed r < and the mean value 
theorem. Furthermore the following limits hold: 

BC A 11 — A 11 

(4.18) hm o D At (r,z,t) = — (r,Af) P,-a.s., Urn t+A ^ * = S 77 (Zf ) P 2 -a.s.. 

The quotient (£n (r, z,t + At) — (n II \r,z,t))/At now takes the form 

(4.19) E, [£ Ai (r, z, t) (A# At - A?) /At] = E z [D M (r, z, t)I {Tn < t} {A^ M - A* 1 ) /At] 

+ E z [D At (r, z, t)I {Tn > t+At} (AH At - A* 1 ) /At] 
+ E z [D At (r, z, t)I {t<Tn<t+At} (AH At - Al 1 ) /At] . 
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Since I{ Tn <t} {^t+At ~ A{ 1 ) /At = ^{T n <t}^ 2 n{Z Tn ), inequality (|4.15p in the claim above, the Dom- 
inated Convergence Theorem, boundedness of D/\t( r , z,t) and (|4.18p imply that the first expectation 
on the right-hand side of (|4.19p converges to E z [^f (r, A( I )I^ Tn < t y'S 2 II (Z Tn )] as At -)• 0. 

The random variable I{ Tn >t+At} {^t+At ~ M. 1 ) /At is bounded by a constant for all At, since, on 
the event {r n > t + At}, the chain Z has not left the finite state space U n n E by the time t + At. 
Therefore, by the Dominated Convergence Theorem, the second expectation on the right-hand side 
of gUD converges to E z [^(r, A^I^tyZj^Zt)] as At -> 0. 

We will now prove that the third expectation on the right-hand side of (|4.19p converges to as 
At — > 0. By decomposing the path of Z n at r n on the event {t < r n < t + At} and applying the 
arguments used in the previous two paragraphs to each of the two parts of the trajectory of Z n , there 
exists a constant C + > such that 



\D A t(r,z,t)\ T 

I {t<T n <t+At} 



A 11 



Al 1 



At 



< E, 



+ E 2 



T n - t 



^ t 1 {t<r n <t+At} 



t + At-T n ^ 2 

7T7 L 'II\^T n )J-{t<Tr l ,<t+At} 



< P z [t<r n <t + At] + E z [^u{Z Tn )I {t<Tn 

<t+At}\ • 

The probability P z [t < r n < t + At] tends to zero as At — > by the claim and ^fi(Z Tn )Iit< Tn <t+At} 
is, for At G (0,1), bounded above by the random variable ^jj(Z Tn )Ii Tn<t+ i\, which is integrable 
by (|4,15p . Therefore, another application of the Dominated Convergence Theorem implies that the 
function £^ is right-differentiable in time. In the case At < 0, analogous arguments to the ones 
described above yield the left-differentiability of Cn^ ■ The limits in ()4.18j) and their counterparts for 
At < imply that the left- and right-derivatives in t of Cn^ coincide and part (jaj) follows. 

For the proof of part (|b|), note that, under the assumption |<7i|(z) ^ \o~2\(z) for all z G E, we have 
Sf(z) = (| <7i | — |o"2| ) 2 (^) > for all z G E. Therefore, a completely analogous argument to the one 
that established the equality in (|4.14p . based on the stochastic time-change A 1 and the fact that the 
laws of the processes (R In , Z n ) and (r + B A i, Z n ) coincide, where B A i denotes the Brownian motion 
B time-changed by the increasing process A 1 , implies the representation of £n given in part (jbj) of 
the lemma. The differentiability of £n follows along the same lines as in part (jaj). The details of the 
arguments are now straightforward and are left to the reader. □ 

Lemma 14.31 shows that the functions and Cn^ solve the HJB equations that correspond to the 
Problems P~9|) and P~10j) . 



Lemma 4.3. Let (i H) and Cn 1 be given by ([4TTT]l - (j4TT2j) 



(J) 



(a) The modulus of the partial derivative \ g r \ is bounded on the set (— oo, — e) x E x (0, oo) for any 
e > and the second derivative in space of £n satisfies 



(4.20) 



2 Air) 



d 2 C 

Q r 2 



-(r,z,t) < for all (r, z, t) G (— oo, 0) x E x (0, oo). 
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If \o~\\{z) 7^ I^K- 2 ) f or all z£E, then the modulus \ ^ r \ is bounded on (— oo,— e) x E x (0,oo), 
for any e > 0, and we have 



(4.21) 



Or 2 



(r, z,t) < for all (r, z, t) G (-oo, 0) x E x (0, oo). 



(b) For any T > the following holds for all r < 0,t G [0, T) and z G E 



(4.22) 



inf 

ce[-i,i] 



£ c (^( Vj T-i) 



r, z 



at 



r,z,T - i) 



(4.23) 
and 



where the function C c (cL H \-, -,T — t)j is defined in (|2.3p . Furthermore, we have 

Ci /7) (r, 2, 0) = 1 /or a// (r, z) G (-oo, 0) x E, 
Ci //} (0, z, t) = for all (z, t) G E x (0, oo) . 
If \ a l\( z ) \ a 2\(z) f or oil z £~E, then for all r < 0, i G [0, T) and z G E we have 



sup 

C6[-1,1] 



C c (C«(-, -,T - t))] (r, *) - -g-(r, *,T - t) 



C^ 7) (r,z,0) = 1 for all (r, z) G (-oo,0] x E, 
(P(0,z,t) = /or a// (z,t) G E x (0,oo). 



Proof. (jaj) Let G(r,t) be as defined in Lemma 14.21 Since n'(x 
normal pdf, we have 

OG 
Or 

d 2 G 



-xn(x), where n(-) is the standard 



(4.24) 
(4.25) 



dr 2 



<r,t) 
<r,t) 



Vt 



n 



r 

2 W 2n 



r 

Vt 

r 

Vt 



<o, 



for all r < 0, t > 0. The derivatives Q~ 



= 1,2, are bounded on (r — e,r + e) x (0,oo) for any 
r < and small enough e > and hence, as in the proof of Lemma 14.21 the Dominated Convergence 
Theorem implies 



in) 



Or 



■(r,z,t) 



OG. , 



i 1 ) 



Q2 C (II) 

and (r,z,t) 



2 G { 
Or 2 



for all r < 0, z £ E, t > 0. Inequality ()4.20p now follows from the inequality in ()4.25p and the 



boundedness of | g r | on the product (— oo, — e) x E x (0, oo) is a consequence of (j4.24j) . Under the 
assumption that |o"i|(.z) ^ \o~2\( z ) f° r an z € E, the properties of the partial derivatives in space 
follow from Lemma 14.21 (jbj) and (|4.24p - (|4.25p along the same lines. 

(|b]) In order to prove that £n satisfies the HJB equation above, define a bounded martingale M n = 
(Ml 1 ) te[o,T]> where 



M, 



7i 



ZZr, 



>T\T t 



r < 0,z G E,t G [0,T], 
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where the process R IIn , started at i?£ /n = r, is given in (|4.6p and the corresponding first-passage time 
Tq(R IIu ) is defined in (|4,5p . The Markov property of the process (R IIn , Z n ) and the definition of (i H) 
in (|4.1ip imply the equality 



(4.26) (W[R* In ,Z?,T-t) = P^[r+{R IIn )>T\F t ]=M i 



n 

t > 



for all r < 0,z G E,t G [0,T]. 

Note that, by (I4.24j) . the modulus |^| is globally bounded on the set (— oo,— e] x (0, oo) for any 
£ > 0. Let r < 0, pick any s 6 (0, — r) and consider the stopped martingale M e = (M t e ) te r 0i ri, defined 
by 

M t e := At , where r+ E := inf{s > : R 1 / n = -e}. 

Ito's formula for general semimartingales (SJ Sec II. 7, Thm. 33] applied to the representation in ([4.26P 
of the martingale M £ , Lemma 14.21 (jaj), Lemma 12.21 applied for the process U = (R I t ^ n + )te[o,T] an d 

the bounded function Cn H \ and the facts that the quadratic covariation [R IIn , Z n,t ] t = vanishes 
for all times t and coordinates Z n ' % of the chain Z n (recall that E C M ), — is bounded on 



(— oo,— e] x E x (0, oo) and P r 
( N t)te[o,T], where 



Ri[ n + < -e, Vt>0 



1 together yield that the process ./V 



E 



N t £ 



tAT_ e+ 



i a 2 c {II) 

-{\a t \ + \a 2 \) 2 (Z?)-^(Rl In ,Z2,T- s) 



ds, 



is a continuous martingale. Hence, since the quadratic variation of N £ vanishes, we have Nf = for 
all times t and starting points (r, z) with r < —e. Since e > is arbitrarily small and the process 
(R IIn , Z n ) visits neighbourhoods of all points in its state space with positive probability, this implies 
that for all r < 0, z € E and t G [0, T): 

(4.27) -(M + |a 2 |) 2 (z)^Hr, z,T — t) + (QCk H) (r, ; T - t))(z) - -^-(r, z, T - t) = 0. 
To prove the first HJB equation above, note that for any c G [— 1, 1] the following inequality holds 

Q2AH) Q2 {II) 

{al-2ca l a 2 + al)(z)-^{r,z,T-t) > (\ ai \ + \a 2 \) 2 (z)-^(r, z,T - t), 

for all r < and z G E, since £ 73&-(r, z, T - i) < by K20| . This inequality, the definition of C C Q ' 
in (I2.3P and identity (I4.27P imply (I4.22p . The boundary behaviour of the function (b H \ stated in the 
lemma, at t = and at r = follows directly from the representation of given in (|4.1ip . 
In the case of the function Cn\ by (|4.2ip it follows that 

-(\ ai \-\* 2 \) 2 (z)^(r,z,T-t) < -(al-2ca 1 a 2 + al)(z)^(r,z,T-t) 

for any c G [—1, 1] and all r < 0, z G E, t G [0, T). An analogous argument to the one in the case of 
(n 1 ^ establishes the HJB equation in (|4.23p and the required boundary behaviour. This concludes the 
proof of the lemma. □ 
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We can now prove that Cn and £n are the value functions for Problems (|4.9p and (|4.10p . 

Lemma 4.4. Pick a time horizon T > and, for any V G V, let R n {V) and Tq(R u (V)) be as in (|4.4p 
and ()4,5p respectively. 

(a) The function Cn > defined in (|4.1ip . satisfies the following: 

C4 7/) (n z, T) = urf Pr,z [r + (ii n (y)) > r] /or any r < 0, 2 G E. 

Assume that \o~\\(z) ^ \o~2\(z) for all z£E. T/ien f/ie function Cn > given in (I4.12p . satisfies 

CP(r, z,T) = sup P r 2 [r+(i? n (y)) > T] /or any r < 0,z £ E. 

vev 

Proof, (jaj) Pick any Brownian motion 76 V and, for any t G [0, T], define the corresponding Brownian 
motion V IInt = (V s IInt ) s > G V by 



(4.28) y, 



lint 



] V s ifa<t, 

[ 7 + 7/ /n - y/ /n if s > £, 

where 7 77n G V is given in (jl.4p with Z substituted by the stopped chain Z n . The Bellman process 
S 177 = (5 , / / )f g [o i T] takes the form 

S? := P r , z [4(R n (V IInt )) > T\F t ] = fa+JV), Z^ M ,T - t) , 

for any r < 0, z G E, t G [0, T], where the second equality follows from the Markov property and 
definitions ()4.1ip and (|4.28p of the candidate value function Cn^ and of the Brownian motion V IIn . 
Note that the stopping time Tq on the right-hand side of the second equality is given by Tq := 
Tq (R n (V)), and hence does depend on the choice of V. This is not explicitly stated in the formula 
for brevity of notation. 

To establish that the Bellman principle applies, it is sufficient to prove that the process is 
a submartingale for any starting point (r, z) of the process (R n (V),Z n ). By Lemma 14.21 (jaj) and 
Ito's formula for general semimartingales Sec II. 7, Thm. 33] we obtain the following pathwise 
representation of the process S 11 : 



£ Ct C { n n) ) W(n Z^T - s) - -^—( R n(y),Z?,T- s) 



S» = C n n \r,z,T) + f T ° 

Jo 

+ Yl [ci II) (^(V),Z^T- S )-Ci n KR^(V),Z-_,T- S 



ds 



4 



* Ar o 

(QCi /7) (^(^),-,T- S ))(Z«_)d,, 



o 



where C = {Ct)t>o is the stochastic correlation process from Lemma 12.11 which corresponds to the 
Brownian motion V , and C c Cn^ is defined in (|2.3p for any constant c G [—1, 1]. This representation 
of S 11 relies on the fact that the continuous part of the quadratic covariation [R n (V) : Z n,l ] t vanishes 
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for all times t and coordinates Z n,t of the chain Z n . Since C\ G [— 1, 1] for all t > 0, equality ()4.22p 
implies that the first integral on the right-hand side is almost surely non-negative and hence we have: 

ft/\T+ O AH) 

(4.29) Si 1 > (i 1 II \r,z,T) + J —^—(R^(V), Z™,T — s) dR™(V) 

+ E [ci 77) ran Zs, t-s)- c { n n) (R:(v), z?_,t - 1 

0<s<tAr,t 







(Q^ n) (R^(V),;T- s ))(zf_)ds. 



Apply Lemma 12.21 with F(s,r,z) := Cn ( r , z,T — s), U := R n (V) and the chain Z n (with bounded 
generator Q n ), to conclude that the process given by the second and third lines of the last display is 
a martingale. 

The first integral on the right-hand side of (14.29P is clearly a local martingale since the integrator 
R n (V) is a martingale. Define a stopping time t^ £ := inf{i > : Rf(V) = —e} for any e > and 
note that t^ £ < Tq . The quadratic variation of the local martingale 

/ J^ m v),Z2,T-s)dR«(V)) 
\ Jo dr /te[o,T] 

is, by Lemma [4.3l ffal) and assumption (II. 2ft . bounded above by an integrable random variable. Therefore 
this stochastic integral is a martingale and, by taking expectations on both sides of inequality (j4.29f) . 
we obtain 

E,, 2 [St\ t +] > CL H) (r, z, T) for all e > 0. 
Since S is a bounded process and the following almost sure limit holds, 



J^ 5 TAr+ £ - S TAr+ ~ / {r+(i?«(V))>T}> 

the Dominated Convergence Theorem implies the inequality 

P r>z [4(R n (V)) >T}> d n \r, z, T) for any V G V. 

This inequality and the definition of C,n^ in (|4.1ip imply part (|aj) of the lemma, 
(jb} For any Brownian motion V £ V and t G [0, T], define V Int = (V s Int ) s > G V by 



V s 



Int ._ 



V s if s < t, 

V t + V s In - V t In if s > t, 



where V In G V is given in (|1.4p with Z n in the place of Z. The Bellman process S 1 = (S 1 / )te[o,T] is given 
by S[ := P r , 2 [r+(R n (V Int )) > T\F t ] = & ] (R^+JV), Z^ M ,T - t) , for any r < 0, z G E, t G [0,T], 
where again Tq := r^ 1 " (i? n (y)) and the second equality holds by the Markov property and (|4.12p . 
Analogous arguments to the ones used in the proof of part (jaj) can now be applied to conclude that 
the process S 1 , appropriately stopped is a supermartingale. Then the optimality of the Brownian 
motion V In G V follows by a limiting argument as in part (jaj) . This concludes the proof of the 
lemma. □ 
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4.2. Proof of Theorem 14.11 We establish Theorem 14.11 in two steps. The first step consists of 
generalising the result of Lemma 14.41 © , 

(4.30) $p(r, z, T) = sup P rz [r£(R n (V)) > T) for any r<0,z£E and n G N, 

to the case where the assumption |<ti|(-z) 7^ |<72|(z) for all z G E is not satisfied. The function in 
this expression is given in (|4.12|) and R n (V) and r Q f '(i? n (y)) are defined in (|4.4p and (|4.5|) respectively. 
The second step in the proof of Theorem 14 . 1 1 consist s of a limiting argument that generalises Lemma f4.4l 
to volatility chains with possibly unbounded generator matrices. 

Consider the case of general volatility functions 01,02 : E — > R, which are only assumed to satisfy 
integrability condition ()1.2|) . Then, for any e > 0, there exists a function cr\ : E — > M that satisfies (jl.2p . 
coincides with a\ on the set where the moduli of the original volatility functions are already distinct, 

{zeE:al(z) = a 1 (z)} = {z G E : faftz) ^ \a 2 \(z)} 

and has the following properties: 

l°ll(*0 T^MO 2 ), ~ cr\{z)\ < e, sgn(a{(z)a 2 (z)) = sgn(a 1 (z)a 2 (z)) for all z G E. 

Note that, in order to define erf, we used the fact that |<ti| + \a 2 \ > 0, which implies that if |o"i|(z) = 
|o"2|(z) for some z G E, then |<ti|(z) > 0. 

Define the process R n,e (V) by (j4.4|) . but with a\ replaced by erf, and note that for any t > we 
have 

(4.31) R^(V) - R?(V) = f [at(Z-) - a^)} dB s . 

Jo 

The chain Z has no instantaneous states by assumption and hence Z n , defined in (|3.3|) . has only finitely 
many jumps, say iVV(Z n ) G NU {0}, during the time interval [0, T]. Therefore identity (|4.3ip implies 
the inequality - R r t l (V)\ < e(l + N T (Z n ))(sup se[ ^ T] B s - inf s , 6[0)T ] B s ,) for all t G [0,T]. 

Since the right-hand side of this inequality does not depend on i G [0, T], the random variables 
s t( v ) := su Pte[o,r] R t ,E ( v ) and St(V) := sup ie[0)T] Rf(V) satisfy 

\SUV)-S T (V)\ < e(l + N T (Z n )) ( sup fi s - inf B s , ] and limSf (V) = SWV) P rz -a.s. 

This implies Zrs T (Y)<o} < liminf e __ >0 ^{S|,(y)<o}- Fatou's lemma and the fact that {/S't(V') < 0} = 
{T+{R n {V)) > T} therefore imply 

P r ,z [r+(R n (V)) >T] < liminf P r , z [S e T (V) < 0] = liminf P r , z [r+(R n ' e (V)) > T] 

(4.32) < liminf P TjZ [r + (i? /n ' e ) > t] , 

where the process R In,e is defined in (|4.6p with o\ substituted by a\ and the last inequality follows 
by Lemma [Q] © • 

Define a strictly increasing process A I,e = (A\' e )t>Q and a non-decreasing process A 1 = (A{) t >Q, 
analogous to (|4.13p . by 

AY:= [\\at\-\a 2 \) 2 (Z?)ds, A\:= f {\o x \ - \<r 2 \f{Z?) &s. 
Jo Jo 
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The properties of <rf imply that A 1 ^ > A\ P^-a.s. for all t > 0. As in the proof of Lemma 14.21 the 
independence of B and Z (by Lemma [2 .3j) implies that the processes (R In ' e , Z n ) and {r + B A i, t , Z n ) arc 
equal in law, where B a i,<l denotes the Brownian motion B time-changed by the precess A I,e . Similarly, 
we have that the laws of (R In ,Z n ) and (r + B A i,Z n ) coincide, where R In is given in (14. 6p . These 
observations imply the almost sure inequality, inf{i > : B.i, t = — r} < inf{i > : B A i = — r}, 

t t 

and the fact that the random variable on the left-hand side of this inequality has the same law as 
Tq (R n ' e (V)) while the one on the right-hand side is distributed as (i? n (V)). This therefore implies 
the inequality 

P r;Z [r+(R In >*) > T] < P r>z [r+(R In )>T] 

which, together with (|4.32p and the definition of in (|4.12p . yields (|4.30p and hence concludes step 
one of the proof of Theorem 14.11 

In the second step of the proof we assume that the volatility process Z is a general (J-j)-Markov 
chain with state space E C M. d , defined in Section [TJ For any n 6 N, in (13. 3D we defined a stopping 
time r n and a chain Z n , which is equal to Z up to the time r n . Lemma 14.41 (jaj), equality (I4.30P and 
the definitions of the functions Cn an d Cri in (|4.11|) - (|4.12[) imply the following inequalities for any 
Brownian motion V € V, 

(4.33) P r , z [r+(R IIn ) >T]< P r , 2 [r+(R n (V)) > T] < P r , z [r+(R In ) > T] , 

where R n (V) is given in (|4.4p and R In ,R IIn are defined in (|4.6p . Furthermore, for any t in the 
stochastic interval [0, r n ] the following equalities hold: 

R?(V) = R t (V), R( In = R t (V n ), R[ n = RtiV 1 ), 

where the process R(V) is defined in ()2.2[) and the Brownian motions V 1 and V n are given in (jl.4p . 
Therefore, we have that, on the event {r n > T}, the random variables I^ T +^(v))>T} anc ^ ^{t + (r(v))>t} 
coincide. The same holds true for the pairs I^ t +^rii^ >t ^ and I{ T +(R(y n ))>T}' an( ^ ^{rf(R In )>T} an< ^ 
^{t|(_r(v 7 ))>t}- Since (T n ) ng N is a non-decreasing sequence of stopping times, such that r n /• oo 
P^-a.s. as n — > oo, we obtain the following almost sure limits: 

/ {r+(i?««)>T} = / {r+(R(y«))>T}' J™, / {r+(i?.^)>T} = 7 {r+ (R(V'))>T} > 

i^-WoW))^} = J {T+(ii(v))>T}' 

These equalities, a final application of the Dominated Convergence Theorem and the inequalities 
in P~33|) imply (|4TT]) - (|4T2|) . This concludes the proof. □ 

4.3. Time-varying extremal couplings. It is tempting to try to prove/generalise the result in 
Theorem l4.1l via a direct argument based on the Dambis, Dubins-Schwartz (DDS)-Brownian motion [9J 
Thm V.1.6], avoiding the Bellman principle. Let X^ 1 ) = (sj 1 ^)^ and S( 2 ) = (Sf } ) 

t>o be two 

progressively measurable processes on (fl, (T^tyo, J 7 , P), such that J^E^H^^j ds < oo for i = 1, 2 
and any t > 0. As usual, for any V E V, define the difference process RiV) = (Rt(V))t>o by 
Rt(V) := r + Jq di? s — dV^, r < 0, i > 0. Let the candidate extremal Brownian motions 
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V 11 = (V t H ) t > and V 1 = (V/) t >o be given by 



(4.34) V t H := - J sgn (x^X^) dB s and V? := ^ sgn (x«X< 2 )) d5 s . 

Under these assumptions the process R(V) is a martingale for each V G V. Hence, by [9j Thm V.1.6], 
there exists a (DDS)-Brownian motion PF^, adapted to the filtration (Fe u {V))u>0-, where the processes 
A(V) = (A t (V)) t > and £(V) = (E U (V)) U > are defined by 

MV) := J* ((X«) 2 - 2C7 S £( 1 )£( 2 ) + (X( 2 )) 2 ) ds and E U (V) := mi{s : A.(V) > u} 

and C = (Ct)t>o is the stochastic correlation between the Brownian motions B and V from (|2.ip in 
Lemma |2. 11 and the following representation holds 

Rt(V) = r + W\ t{v) for all* > 0. 

It is clear from these definitions that the following inequalities hold almost surely for all times t > 0: 

(4.35) A\ := jf* (|S«| - |S( 2 )|) 2 d S < A t (V) < jT (|S«| + |Si 2 )|) 2 d S =: ^. 

Let 7g~(.R(V)), rj" (r + W^r/ ) and Tq (r + W^ I ) denote the first-passage times over zero of the processes 
R(V), r + WXij and r + WYi, respectively, and note that the inequalities in ()4.35p imply 

(4.36) r+(r + Win) < r+(R(V)) < r+(r + W V AI ) 

on the entire probability space Q, for every Brownian motion V G V. 

It is tempting to conclude from this that the processes r + WYj-/ and R(V n ), where the Brownian 
motion V is defined in ()4.34p have the same law (ditto for the pair r + Wl and RiV 1 )), which 
would together with (|4.36p . yield a generalisation or an alternative proof of Theorem 14.11 However, 
the counterexample in Section 11.2.11 demonstrates that the generalised mirror coupling in ()4.34p can 
be suboptimal in this setting. The counterexamples to Theorem 14. H based on the continuous-time 
Markov chains in Section I5T2"| which are adapted non-Mar kovian processes with respect to the filtration 
(•^ r t)t>0) clearly show that this approach cannot be used as an alternative proof of Theorem H7TJ because 
it only requires the volatility processes to be (J r t)-adapted. We should stress here however, that in 
the case of deterministic integrands ZK 1 ) and Y,( 2 \ Proposition 14.51 can be established. 

Proposition 4.5. Let S^),S^ 2 ) be deterministic processes (i.e. measurable functions of time) that 
satisfy the integrability condition above, |Ss |, > for all s > and Af. 1 \A\ /* oo as t /* oo. 

Then for any time horizon T > and Brownian motion V G V , the following inequalities hold: 

P r [r+(R(V n )) > T] < P r [tq(R(V)) >T}<P r [^(R^ 1 )) > T] . 

Proof. The integrability assumption J^X^^ds < oo, i = 1,2, from the beginning of Section 14.31 
implies that A 11 is a well-defined, finite, strictly increasing differentiable function. Its inverse E n , 
which is defined on [0, oo) since the limit A 11 tends to infinity with increasing time, is also strictly 
increasing and differentiable and satisfies the following ODE: 

(4-37) E?= /7|X« l + lE^fT'ds. 



%e would like to thank David Hobson for this observation. 
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Since the left-hand side of (|4.37p is finite for all u > 0, for any V € V the process W IIV = (W^ IV ) 



t>o, 



(4-38) Wr ■= jf (|E<§,| + dWX, 

is well-defiend for all t > 0, where denotes the (DDS)-Brownian motion introduced above. The 

quadratic variation of the continuous local martingale W IIV is by (I4.37P equal to = E I h I = t, 

A t 

making W IIV a Brownian motion by Levy's characterisation theorem. By (14.38P we obtain dW^Y = 
dWf /(|£«| + |E<g,|) and W* = J U (|S« | + \^%\)dW^ = f*\\V?>\ + \^\)dW s IIV , where 
the last equality follows by Prop V.1.4]. Hence we find W\ u = JJflE^I + |si 2) |)dW/ /y for 
all t > 0. Since X^ 1 ) and S^ 2 ^ are non-zero everywhere by assumption, the process W = (Wt)t>o, 
given by Wt := Jq sgn(T,^)dWg IV , is a Brownian motion and the equalities (si 1 ^ = sgn(S^)si 1 ^ 
and sgn(S^Ei 2 ^) = sgn(si 1 ^) sgn(Es 2 ^) hold. Therefore, the processes R(V H ), where V 11 is given 
in (|4.34|) . and r+WYn are equal in law and hence (|4.36p implies the first inequality in the proposition. 
The second inequality follows along the same lines. □ 

Remarks. (i) It is important to note that the Brownian motion W IIV , introduced in (|4,38p . is not 
an element of the set V as it is in general not adapted to the original filtration (Ft)t>o- In fact, 
W IIV is an (J 7 t )-Brownian motion only in the case V = V n . 
(ii) The final step in the proof of Proposition 14.51 relies on the fact that the stochastic integrals 



L 



r ( S W + sgn (E«Ei 2 )) Si 2 )) dB s , I (SW + sgn (x^) E®) sgn dW» V , 

where B is a fixed Brownian motion and W IIV is defined in (|4.38j) . are equal in law, which holds 
since £W and S^ 2 ) are deterministic. Assume that both processes non-deterministic, 
but adapted to (J 7 t)t>o and independent of the Brownian motion B. Then, it is not clear whether 
one can define the second stochastic integral, since W IIV is not (J^)-Brownian motion. Even if 
this were possible, the laws of the two integrals would in general not coincide since the integrand 
and the integrator are independent in the former and dependent in the latter integral. 

5. Counterexamples 

5.1. The presence of drift. If either of the processes X and Y(V) in (jl.ip can have drift, the 
conclusion of Theorem 11.21 fails as the following example demonstrates. 

Let R(V) be the difference of X and Y(V) and assume that it takes the form 

R t (V) = r + fit + B t - aV t , 

where B is the fixed (J^-Brownian motion, V E V an arbitrary (J^)-Brownian motion, a a volatility 
parameter different from 1, r a strictly negative starting point and \x a constant positive drift. Then 
the candidate extremal Brownian motions in (jl.4j) are given by V 1 = B and V 11 = —B and the 
following lemma holds. 

Lemma 5.1. For any fixed values of r < 0, T > 0, a > 0, there exists a large positive drift fi such 
that the inequality P r [r + (i?(y 7 )) > T] < P r [t+(R(V h )) > T] holds. 



2H 
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Lemma [5,11 implies that Theorem 11.21 cannot hold for processes with drift. An intuitive explanation 
for this phenomenon is as follows: in the presence of a large drift upwards, it is better to reduce the 
volatility as much as possible (in this case to the level |1 — a\), instead of increasing it to its maximal 
value (equal to (1 + a)), since the drift makes the processes X and Y(V) couple before time T . 

Proof. Fix r < 0, T > 0, a > 0, li > and define the function F : (0, oo) -)• [0, 1] by 
F(„) := N (- r -±fv) - (_L_^„) . „ > „, 

and recall that P r [t^R^ 1 )) > T] = F (1/|1 - ct|), P r [t+(R(V h )) > T] =F {!/{! + a)) (see e.g. [21 
II.2.1, Eq. 1.1.4]), where N(-) denotes the normal cdf. To establish the lemma it is sufficient to find 
li > such that the corresponding F is strictly decreasing on the bounded interval [1/(1+<t), 1/|1 — ct|]. 
It is easy to see that for any a < it holds N(a) > —n(a)a/(l + a 2 ), where n(-) is the normal pdf. 
After a slightly tedious calculation and an application of this inequality, it follows that there exists a 
real constant C, which depends on our bounded interval, such that the inequality holds: 

F*(v)<n(^j=ty (^f + f) fora11 «€[1/(1 + ?),1/|1-?|]. 

There clearly exists a large li > 0, such that the right-hand side of this inequality is strictly negative 
on the entire interval (recall r < 0) and the lemma follows. □ 

5.2. Markov chains as (J r t )-adapted but non-(J 7 t )-Markov processes. In this section we con- 
struct two continuous-time Markov chains on our probability space, both of which are (J r t)-adapted 
but do not satisfy the Markov property with respect to the filtration (Ft)t>o- We show that in each 
of the two examples, the strategies in Theorems 11.11 and 11.21 are suboptimal. This demonstrates that 
the assumption that the chain Z is an (J r t)-Markov process is indeed necessary. 

5.2.1. Time-inhomogeneous case. Recall that B is (J^)-Brownian motion, fix e £ (0, 1) and then let 
the random times T n , n G N U {0}, be given by To := and 

T n := inf{t > T n _i : \B t - B Tn ^ | = e} for n > 1. 

Define the processes = (N t )t>o and W = (Wt)t>o by 

N t := max{n G N U {0} : T n < t} and W t := B Nt . 

For every t > we have {T n < t} £ Ft for all n G N and hence the process W is (J r t)-adapted. 
Furthermore, in its own filtration, W is a continuous-time time-inhomogeneous simple symmetric 
random walk on eZ. In particular, W has only finitely many jumps on any compact interval. Let 

Z := zo£(W), for a fixed zq > 0, 

where £ denotes the Doleans-Dade stochastic exponential [5J Sec II. 7, Thm. 37]. Therefore, by defini- 
tion, we have 

ft rT Nt 

Z t = z + Z s _ dW s = z + Z s _ dW s = Z TNt , 
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where the second equality follows from the facts that T/v t < t, and that there are no jumps of W 
during the time interval (Tjv t ,i]. The process Z has a countable state spaced which can be expressed 
as E := {zq(1 — e ) n (l + e ) m : m,n G N} C (0, oo), is a time-inhomogeneous Markov chain (since its 
holding times are not exponential) with respect to the filtration generated by Z and is non-Markovian 
with respect to (Tt)^- Furthermore, it is clear from the construction that the pair (B,Z) is (J-'t)- 
Markov. 

Consider the stochastic integral j Q Z s dB s and note that the equality Wx n — Wt„- = Bt u — Bt„^ 1 
holds for all n G N. Hence the stochastic integral can be expresses as follows: 
rt pT Nt M 

/ Z s dB s = ZsdBs + ZsdBs = Z T n -! (B Tn - B Tn _,) + Z TNt (B t - B Tfft ) 

Jo Jo Jr Nt n=1 

= (Z TNt - zo) + Z TNt (B t - B TNt ) = Z t (l + (B t - B^)) - z . 
Therefore, since by definition we have \Bt — Bt n \ < e and Zt > 0, the following inequalities hold: 

(5.1) -Zq<(1- e)Z t -z < f Z s dB s for all t > 0. 



As in Section [1.2.11 define a% : E — > R by a{(z) := — iz for any z£E and i = 1,2, and note that 
by (jl.4p we have V 1 = B and V = —B. Hence, for any starting points i,j/6l, definition (jl.ip and 
inequality (|5.ip yield the following almost sure inequalities: 

X t -Y t {V I ) = x-y+ [ Z s dBs>x-y-z , X t - Y t {V H ) = x - y - 3 f Z s dB s < x - y + 3z . 

Jo Jo 

For any time horizon T > 0, counterexamples to the Conjecture in Section Tl. 21 (for both Problems (T) 

and (C)) can now be constructed in the same way as in Section [1. 2. li 

5.2.2. Time-homogeneous case. In order to define a process Z, which is a time-homogeneous Markov 
chain in its own filtration and has properties analogous to the ones in the previous section, we sample 
the path of the Brownian motion B at a sequence of independent random times. 

Fix a small e > and let (e n (e)) n6 N be a sequence of independent exponential random variables on 
the initial probability space with E[e n (e)] = e for all n £ N. Define a sequence of random times T n , 
n G N U { — 1,0}, and a Poisson process N = (Nt)t>o by 

T_! := T := 0, T n := T n _ x + e n (e) for n G N, and N t := max{n G N U {0} : T n < t}. 

Furthermore, define the function g e : R — > R by g e (x) := m&x{y G eZ U {e2~ n : n G N} : y < x} 
and note that g e (x) = t[x/e\ for all x G R\ (0, e), where [y\ denotes the integer part of y G R. The 
function g t will help us discretise the increments of the Brownian motion B over the time intervals of 
lengths e n (e). The finer resolution in the interval (0, e) via g e is necessary, so that the corresponding 
chain W e (defined below) has a non-zero jump at every random time T n . 
We can now define the process W € = (Wl)t>o by 

N t 

Wt-Y^deiB^-BT^). 

n=0 



2 An additional bijection is needed to define a chain with a state space that is a discrete subspace of a Euclidean space. 
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Note that W e has a countable state space, jumps only finitely many times on any compact time interval, 
has exponentially distributed holding times and, due to the independence of B and (e n (e)) ne N> the 
jumps Wl — Wl_ = g e (^BT Nf — Bx Nf J are distributed as g e (f? ei ( e )) for all t > 0. Therefore W e is 
a time-homogeneous Markov chain and so is its stochastic exponential Z e := zo£(W e ) (see e.g. [U 
Sec II. 7, Thm. 37] for definition). It is clear from this construction that for any T > we have 

r T 

lim / (Zl - Z t f dt = P Z0 -a.s., 
Jo 

where Z is defined in (11 .5f) . Therefore, since the stochastic exponentials Z and Z £ are square inte- 
grable, the Burkholder-Davis-Gundy inequality [U Sec IV. 4, Thm. 48] implies the following almost 
sure convergence 

<t>( X T ~ Y t( V )) ~> 4>( X T - Yt{V)) , I{ T0 ( X c-Y"-(V))>T} ->■ I{t q (X-Y(V))>T}, 

as e — )• 0, for any Brownian motion V G V, cost function and volatility functions 01,02 given 
in Section 11.2.11 where the processes X e ,Y £ (V) are defined in (jl.ip with Z replaced by Z e and the 
stopping time tq(X € — Y e (V)) is equal to inf{t > : X\ = Y t e (V)}. The counterexamples from 
Section Tl. 2. II (with a bounded 4>) show that the conjecture in Section [L2] fails (for both Problems (T) 
and (C)) in the case of the process Z e if e > is small enough. 

5.3. (J-f )-Feller process Z independent of B. The final counterexample shows that the "tracking" 
part of the conjecture in Section H. 21 fails for general Feller processes even if Z and B are independent. 

Assume that there exist an (J-^-Brownian motion B 1 - S V, independent of B, and define the (J-t)- 
Feller process Z := zq + B 1 - with state space E := M for any starting point zq G M.. Let (J\(z) := 2z 
and &2(z) := z, for any zGl, and note that by (|l-4j) we have V = B. We will now show that, for 
the cost function <p(x) := x 4 , the first inequality in Problem (T) fails, i.e. there exists a Brownian 
motion V G V such that for any T > it holds 

(5-2) E r , Z0 [(Rt(V))*} < E r , Z0 [(i?T(^)) 4 ] , 

where R(V) = X - Y(V) (and X, Y(V) given in jH} for any V £ V) and i2o(V) = r, Z = z . 

To construct such a process V, define the family V c = (V t c )t>o, c G [—1,1], of (J^-Brownian 
motions by 

V t ° := Vl~c 2 B t + cB t L , 
and note that V° = B = V 1 . Therefore the difference process R(V C ) takes the form 

Rt(V c ) = r + J (2Z s dB s - Z s dVs) = r + (2 - y/l - c 2 ) J^Z s dB s -cJ^ Z s dB^, 
and hence we find d[R(V c ), R(V c )] t = (5 - iVT^c?)Z?dt and d[R(V c ), Z] t = -cZ t dt. 
Lemma 5.2. Define ip c (r,z,t) := E rjZ [(Rt (V c )) 4 ] for any r, z G M and t > 0. T/ien we /lave 
^ c (r,z,t) = r 4 + 6k(c)r 2 z 2 t + 3/c(c)(r 2 + /c(c)z 4 - 4crz 2 )t 2 

+Jfe(c)((7Jfe(c) + 8c 2 )z 2 - 4cr)t 3 + (7fc 2 (c)/4 + 2c 2 fc(c))t 4 , 

where k(c) := 5 — Ay/1 — c 2 for any c G [—1, 1]. 
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Proof. The representation in the lemma for the expectation ip c (r,z,t) follows from martingale ar- 
guments and stochastic calculus. Alternatively to verify the lemma, one can easily check that the 
function 93, given by the formula above, satisfies the PDE 

2 K[ - C > Z dr 2 cz drdz 2dz 2 8t' 

with boundary condition <p(r, z, 0) = r 4 and polynomial growth in r and z. An application of the 
Feynman-Kac formula then yields ip c = (p. □ 

Note that k'(0) = and hence the derivative in c at c = of the value function ip c (r, zq,T) equals 

dip c 



dc 



■{r, z ,T) 



= -r(12^T + 4T 3 ). 

c=0 



Since this quantity is non-zero for any r ^ 0, inequality (|5.2p is satisfied (by Lemma I5.2|) for some 
V = V c with c 7^ (recall that V° = B = V 1 ). An analogous argument can be used to show that the 
second inequality in Problem (T) also fails in this setting. 



Appendix A. Proofs of Lemmas I2TT1 and I2T2I 



A.l. Proof of Lemma l2.ll It is clear that Lemma 12 . 1 1 follows from and the basic properties of 
stochastic integrals if, for any V G V, we can find a progressively measurable process C and W E V, 
such that — 1 < Ct < 1 for all t > P-a.s., W and B independent and 



(A.l) 



V t = f C s dB s + f\l - Cl) 1 ' 2 dW s . 
Jo Jo 



By the Kunita-Watanabe inequality [H Sec II. 6, Thm. 25], the signed random measure d[F,B]t 
on the predictable a-field is absolutely continuous with respect to Lebesgue measure d[B,B] t = dt. 
Hence, there exists a predictable process C = (Ct)t>o, such that d[V, B] t = Ctdt, and for any s < t 
we have |[V,S]t — [V, f?] s | < t — s. Therefore, we may assume that \Ct\ < 1 and define the processes 
D t := (1 - Cl) 1 ' 2 and M t := V t - J * C s dB s . Note that the equalities [M, B] t = 0, [M, M] t = f* D 2 ds 
and Jq I{D s>0 }D~ 2 d[M, M] s < 1 hold. Therefore the continuous local martingale W, given by 



Wi 



t ■- 



f 

Jo 



{D 3 >0} 



d-MMs + 



Jo 



I{D B =o}dB s , 



is well-defined, where B 1 - G V is a Brownian motion independent of B. Levy's characterisation theorem 
applied to W now yields the representation in (|A.ip and hence implies Lemma |2. 11 □ 



A. 2. Proof of Lemma 12.21 The assumptions on Q and F imply that E[|M t |] < 00 for all times 
t > 0. The additive structure of the process M u implies that it is sufficient to prove the following 
almost sure equality: 



(A.2) 



J2 [F(s,U s ,Z s )-F(s,U s ,Z s ^)} 

t<s<t' 











Ft 


= E 2 


^ (QF(s,U s ,-))(Z s _)ds 


Ft 
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for any < t < t' and z £ E. The jump-chain holding-time description of the continuous-time chain 
Z and the continuity of the process U and function F imply 



(A.3) E 2 



[F(s,U s ,Z s )-F(s,U s ,Z s 

u<s<u+Au 



3~ u 



Au(QF(u,U u ,-))(Z u ) + o(Au) 



for any u > and small An > 0. In this expression, for each Au, o(Au) represents an J^-measurable 
random variable which is bounded in modulus by CAu, for some constant C > independent of Au 
(here we use assumption (j2.4|) and the boundedness of F), and limAn^o ° = almost surely. 

We now decompose the left-hand side of (|A.2|) into a sum over the time intervals of length At > 0, 
where £ N, and apply ()A.3j) to each summand: 



[F(s,U s ,Z s )-F(s,U s ,Z s _)] 



t<s<# 



8=0 



[F(s,U s ,Z s )-F(s,U s ,Z s _)] 



(A.4) 



^^ + At Yl E z [(QF(t + iAt,U t+iAt ,-))(Z t+iAt )\T t ) 

i=0 



The properties of the random variables o(At) listed in the paragraph above, the Dominated Con- 
vergence Theorem applied to the right-hand side of (|A.4p as At J, 0, the definition of the Lebesgue 
integral and the fact that Z jumps only finitely many times during the time interval [t, t'\ together 
imply the equality in (|A.2j) • This concludes the proof of the lemma. □ 
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